fix(*): resolve the latest release under windows powershell 5.1 by LivXue · Pull Request #863 · EverMind-AI/Raven

LivXue · 2026-10-05T14:55:09Z

Summary

Native Windows installs fail from the shell Windows ships with. Both commands the README gives for Windows break under Windows PowerShell 5.1:

irm https://raven.evermind.ai/install.ps1 | iex stops with (308) Permanent Redirect. PowerShell 5.1 follows 301, 302 and 307 redirects but not 308, which is what that URL answers with. This is the error fix: cannot install Raven in Windows via powershell 5.1 #148 reported; the README already offered the direct URL for it.
The direct URL downloads the script, which then stops at "Could not resolve the latest Raven release wheel from GitHub". Resolve-RavenLatestVersion requests the release page without following the redirect and reads the Location header off the exception that raises. Only PowerShell 7 attaches the response to that exception. PowerShell 5.1 returns the unfollowed redirect as the response and reports MaximumRedirectExceeded as an error with no response, so the lookup always came back empty. This came in with fix(*): resolve the latest release without the github api quota #299 and has been on main since.

Changes:

install.ps1: the release-page request uses -ErrorAction Ignore instead of -ErrorAction Stop, so PowerShell 5.1 reads Location from the returned response. PowerShell 7 still raises regardless of -ErrorAction and goes through the existing catch. The Location is still matched against the same stable-tag pattern.
CI: the installer job ran install.ps1 under pwsh only, which is how this shipped. It now also runs the piped install under Windows PowerShell 5.1 (shell: powershell), into its own tool, bin and home directories, sharing the uv cache.
Docs: README, README.zh-CN, the docs-site quick start (en, zh) and the release notes template now say that Windows PowerShell 5.1 stops on the short URL with (308) Permanent Redirect and to switch to the direct URL when that appears. The short URL's redirect itself is unchanged.
Tests: tripwires for the lookup (-ErrorAction Ignore, catch kept) and for the CI gate (both shells, and separate tool, bin and home directories, since the version check runs the raven.exe in the bin directory).

Second topic, Windows test hygiene: three tests failed when the suite ran on a Chinese-locale Windows checkout. CI runs the unit tests on Linux only, so it never saw them.

test_no_workflow_step_enters_the_removed_page_directory read workflows in the locale encoding (GBK there) and stopped on the UTF-8 in release.yml. It now reads UTF-8.
The font and LibreOffice harness tests run install.sh code under sh, which on Windows is Git Bash: its curl sits outside /usr/bin and paths mix separators, so two failed and others passed for the wrong reason (the digest-mismatch test passed because curl was missing). The ten harness tests now carry the POSIX-only skip the other three sh-driven tests already had, shared as one POSIX_SH_ONLY marker. Linux runs are unchanged.

#861 merged first, and this branch is rebased onto it. The one conflict was in tests/test_install_script.py. The sh-driven tests #861 added, its two Node harness tests and test_a_download_that_cannot_be_hashed_is_not_installed, now carry @POSIX_SH_ONLY like their neighbours, so the marker stays the one spelling of that skip.

Third topic, test infrastructure: Python's locale-dependent text I/O uses the system code page when no encoding= is passed. On a GBK-locale Windows checkout, any test that reads or writes a file containing non-ASCII bytes will fail or silently corrupt data. The class of bug was found above; there are ~790 read_text() call sites without encoding in tests/. Measured on Linux under a zh_CN.GBK locale, main without UTF-8 mode is not one test short: 87 tests fail only there, across 25 files, and tests/test_rpc_schema_match.py (418 tests) does not collect, mostly on UnicodeDecodeError from reading UTF-8 files.

Makefile: export PYTHONUTF8 := 1, so every recipe runs in UTF-8 mode, check-core-wheel (part of make ci) included.
tests/conftest.py raises at import time on any interpreter whose default text encoding is not the UTF-8 codec, telling the caller to set PYTHONUTF8=1 or use -X utf8. It compares codecs rather than names, so Windows with the system-wide UTF-8 option, which reports cp65001, passes. uv run pytest, which AGENTS.md documents and which does not read the Makefile, now stops here on a GBK checkout, as does an IDE runner that does not set the variable. CI's --noconftest self-upgrade job (its tests all name encoding explicitly) is unaffected. tests/test_conftest_utf8_guard.py pins the codec comparison and drives the refusal in an ASCII-locale child.
CI sets PYTHONUTF8 in no job. The jobs that load tests/conftest.py run on ubuntu-latest, where the locale is UTF-8 already, and the unit job gets the variable from the Makefile through make coverage-shard. The Windows self-upgrade job runs its one file with --noconftest, and that file names its encodings. The installer job runs no tests at all: its steps are raven installing itself and answering --version, so the variable there would only change how raven runs, and that gate exists to run it the way users do.

Type

Verification

Run on Windows 11 (zh-CN) with Windows PowerShell 5.1.26100 and PowerShell 7.6.6, before the rebase onto #861 and the second review round. Every install below was isolated: UV_TOOL_DIR, UV_TOOL_BIN_DIR and RAVEN_HOME in a temp directory, RAVEN_MINIMAL=1, RAVEN_NO_LAUNCH=1.

Before the fix, under 5.1: irm https://raven.evermind.ai/install.ps1 fails with (308) Permanent Redirect; irm https://raw.githubusercontent.com/EverMind-AI/Raven/refs/heads/main/install.ps1 | iex stops at Could not resolve the latest Raven release wheel from GitHub.
Invoke-WebRequest https://github.com/EverMind-AI/Raven/releases/latest -MaximumRedirection 0 -UseBasicParsing -ErrorAction Stop: 5.1 throws InvalidOperationException (MaximumRedirectExceeded) with no Response; 7.6.6 throws HttpResponseException whose Response.Headers.Location is the tag URL. With -ErrorAction Ignore, 5.1 returns the 302 with Location set, and 7.6.6 still throws into the catch.
Redirects under 5.1, via httpbin redirect-to: 301, 302 and 307 are followed; 308 is not.
After the fix, Get-Content install.ps1 -Raw | Invoke-Expression under 5.1 and under 7.6.6, with uv tool update-shell disabled in a temp copy of the script to leave the user PATH alone: both resolved v0.2.4, installed raven with everos-memory, design-engine and ppt-engine, and raven.exe --version printed Raven v0.2.4.
uv run pytest tests/test_install_script.py tests/test_release_plugin_list.py tests/test_constraints_export_contract.py plus the six installer tests in tests/test_cli_onboard_commands.py, on Windows: 48 passed, 13 skipped (before: 55 passed, 3 skipped, 3 failed). The two new tripwires fail against the previous install.ps1 and ci.yml. With the UTF-8 gate added, a bare uv run pytest --co -q tests/test_install_script.py on the GBK checkout raises RuntimeError naming PYTHONUTF8=1, while the same invocations with the variable set pass (45 passed, 13 skipped).
uv run ruff check and uv run ruff format --check on both test files pass; ty does not cover tests/, the only Python this changes. PYTHONPATH=. uv run python scripts/check_source_language.py origin/main exits 0.

Run on Linux (Ubuntu 22.04, Python 3.12.13) at this head. A zh_CN.GBK locale compiled into a private LOCPATH gives a non-UTF-8 interpreter (locale.getpreferredencoding(False) is GBK, UTF-8 mode off).

Full suite, uv run --frozen --python 3.12 --all-extras pytest -q: 7 failed, 27567 passed, 119 skipped. On main the same command gives 7 failed, 27556 passed, 119 skipped with the identical 7 failing IDs, this machine's local baseline (five proxy tests, a root-only permission test and an npm found on PATH), so 0 introduced; the 11 extra passes are this PR's new tests.
Main under GBK without UTF-8 mode: 86 failed, 27059 passed, 119 skipped, 11 errors. 87 IDs fail only there, and tests/test_rpc_schema_match.py does not collect. One of the 87, a non-ASCII file name, is specific to Linux, where GBK also becomes the filesystem encoding.
This head under GBK: a bare uv run pytest --co -q tests/test_install_script.py stops at the conftest RuntimeError naming PYTHONUTF8=1. make test-python, which exports it: 7 failed, 27567 passed, 119 skipped, the same 7 IDs as under UTF-8. make check-core-wheel: 1 passed, where with the per-target exports it exited 2 at the guard.
The harness tests marked @POSIX_SH_ONLY run on Linux in the full suite above.
Mutations: with the old name tuple back in the guard's predicate, only the cp65001 case of tests/test_conftest_utf8_guard.py fails; with the guard deleted, its end-to-end refusal test fails. Giving the 5.1 step pwsh's UV_TOOL_BIN_DIR, or its RAVEN_HOME, now fails the CI gate test naming the shared variable, where before it passed.
make lint-python passes, and the commit-message, source-language and large-file gates exit 0 over origin/main..HEAD.
Relevant tests pass locally
Relevant lint / type checks pass locally
User-facing docs or screenshots are updated when needed

Risk

User-visible: the one-line Windows install works again from Windows PowerShell 5.1 through the direct URL. The short URL still answers with a 308 there, and the docs now name that error. install.ps1 is served from main, so the fix reaches users on merge, and reverting the squash commit rolls it back the same way. The resolved Location is still matched against the same stable-tag pattern before anything is downloaded. The installer CI job gains one Windows install, which reuses the uv cache.

Developer-visible: on a Windows checkout whose code page is not UTF-8, a pytest run that loads tests/conftest.py without UTF-8 mode stops at import with a message naming PYTHONUTF8=1; the make targets set it, and Linux and macOS runs are unaffected. On Windows the sh-driven harness tests report as skipped: the ten this PR marks and test_a_download_that_cannot_be_hashed_is_not_installed, beside the five that already skipped there.

Security impact considered
Backward compatibility considered
Rollback path is clear for risky changes

Related Issues

#148

gloryfromca

No blockers; this can merge as far as I am concerned.

Reviewed github/main...HEAD at d57ad789ea28. The release lookup now handles Windows PowerShell 5.1's non-terminating redirect error through the returned response while preserving PowerShell 7's exception-response path. The added Windows CI step exercises the stock shell independently from pwsh, and that installer job passed. The POSIX harness skips do not weaken Linux CI coverage; they prevent Git Bash from producing host-inaccurate results.

I covered the repository rules, the complete diff, relevant callers and history, backward compatibility across both PowerShell families, test-strength changes, and architecture constraints (no domain or layer boundary is affected). Verification: uv run --frozen --python 3.12 pytest tests/test_install_script.py tests/test_cli_onboard_commands.py -q passed with 395 tests; the source-language and large-file check scripts both exited 0.

0xKT · 2026-10-05T15:14:27Z

Not a blocker -- the change is right and I am not asking for anything to

Severity of this one finding, not a verdict on the pull request. The blocking findings from this pass are review threads on the changed files; GitHub renders those collapsed, as a file name with no text.
move. One piece of merge-order information the board can see and a single PR
cannot, plus one note on the scope of the encoding fix.

This PR and #861 each merge cleanly onto the tip, but conflict with each
other. install.ps1 auto-merges -- you touch Resolve-RavenLatestVersion
and #861 touches Install-PrivateNode -- but
tests/test_install_script.py does not:

merge-tree 861 x tip   -> clean
merge-tree 863 x tip   -> clean
merge-tree 861 x 863   -> CONFLICT (content): tests/test_install_script.py
                          (install.ps1 auto-merges)

The cause is that you are both editing the same decorator lines from opposite
directions. #861 adds four tests, each carrying an inline
@pytest.mark.skipif(sys.platform == "win32" or shutil.which("sh") is None, ...);
this PR replaces every inline copy of that decorator with the shared
@POSIX_SH_ONLY. Whichever lands second needs a hand resolution.

Worth knowing beyond the textual conflict: the shared marker exists precisely so
there is one spelling of "POSIX sh only", and a resolution that simply keeps both
sides leaves #861's two new sh-driven tests carrying inline decorators while
everything around them uses the marker. Landing this PR second, and converting
those two as part of the resolution, is the order that ends with one definition.
Entirely the maintainer's call -- I am only reporting that the pair needs a
decision, since neither PR can see the other.

The UTF-8 fix addresses the instance, not the class. path.read_text() ->
read_text(encoding="utf-8") in test_cli_onboard_commands.py:6496 is correct
and is the call that actually failed. For the record, the same shape is
everywhere: grep -rn "read_text()" tests/ | grep -v encoding= is 790 call
sites. Almost none can fail today -- most read JSON the test just wrote, or
ASCII -- and they only bite on a non-UTF-8-locale Windows checkout reading a
file that happens to contain non-ASCII, which is exactly the one you hit.

I am not asking you to touch the other 790; that is a different change and
charging this PR for it would be charging the wrong one. But there is no owner
for the question today -- no PYTHONUTF8 in pyproject.toml, the CI workflow or
tests/conftest.py -- so the next person on a GBK console finds the next one by
having it fail. One PYTHONUTF8=1 in the test environment would retire the whole
class, if you think it worth a follow-up.

For what it is worth, the part of this PR I found most valuable is the admission
in test_the_ci_gate_runs_the_windows_install_under_both_powershells: the gate
was green while every install from the stock shell failed. Adding the second
shell to CI is what stops that recurring, and it is a better fix than the
one-line -ErrorAction change it accompanies.

gloryfromca

No blockers; this can merge as far as I am concerned.

Reviewed the delta from d57ad789ea28 and rechecked the resulting github/main...HEAD diff. The new PYTHONUTF8=1 configuration covers every CI job that invokes pytest, is inherited by the intended subprocesses, and does not skip or weaken any tests. The Windows installer and self-upgrade jobs pass on this head.

I rechecked the repository rules, affected workflow callers and history, backward compatibility, test integrity, and architecture constraints; this CI-only delta does not cross a domain or layer boundary. Verification: uv run --frozen --python 3.12 pytest tests/test_install_script.py tests/test_cli_onboard_commands.py -q passed with 395 tests, and the source-language gate exited 0.

0xKT · 2026-10-06T07:51:34Z

Not a blocker. Round two on 042a7e58. The delta is one commit touching only

Severity of this one finding, not a verdict on the pull request. The blocking findings from this pass are review threads on the changed files; GitHub renders those collapsed, as a file name with no text.
.github/workflows/ci.yml, and I re-read it line by line. Two nits, neither holding
anything; the rest checks out and your own description is accurate where I checked it.

Gates on the merged tree (base is an ancestor of head, so head is the merged tree):
gates.sh ruff/lint-imports/commit/large/lang rc=0; make check-large-files rc=0;
tests/test_install_script.py + tests/test_cli_onboard_commands.py -> 395 passed in 10.50s
(same 395 as round one, which matches a delta that adds no test); the other three files that
read a workflow (test_constraints_export_contract, test_dependabot_scope_canon,
test_release_plugin_list) -> 14 passed; non-ASCII added lines live only in
README.zh-CN.md (1) and docs-site/docs/quick-start.zh.md (2).

What I checked and found right

Your "On Linux this is a no-op" is correct, and I verified it rather than taking it: the
unit job's matrix is os: [ubuntu-latest] and trajectory is runs-on: ubuntu-latest,
so two of the three placements never see a non-UTF-8 runner. On a UTF-8 locale
locale.getpreferredencoding(False) is UTF-8 with or without the variable (only
sys.flags.utf8_mode moves: 0 -> 1), so nothing those jobs read decodes differently.
windows-upgrade is the one placement on a non-Linux runner.

The class you describe is real and I measured it at this head: 792 read_text() call sites
with no encoding= under tests/, 2620 non-ASCII text files in the repo, and 575 of those
cannot be decoded as cp1252 at all -- including the SKILL.md that
tests/_ppt_engine_skill_claims.py:25 reads at import time, with no encoding=. The
release.yml case you fixed is four em-dashes (e2 80 94), which cp1252 decodes and GBK
rejects -- so that test failed on your machine and could not have failed on CI.

Nit 1 -- the measure is in the one place the developer it names never reads

Your stated beneficiary is "the next GBK-console developer", and the variable is in
ci.yml, which a local uv run pytest never executes. git grep PYTHONUTF8 outside that
file returns nothing; there is no -X utf8 and no UTF-8 setting in pyproject.toml or the
Makefile, so a local run on a GBK box is exactly as exposed after this commit as before.

The three placements, measured off the parsed workflow:

job	runs-on	PYTHONUTF8
`unit`	`ubuntu-latest` only	job-level, no-op
`trajectory`	`ubuntu-latest`	step-level, no-op
`windows-upgrade`	`windows-latest`	step-level; that test does no encoding-less text read
`installer`	`ubuntu-latest` and `windows-latest`	not set

So nothing it is on can change a decode today, and the only job whose matrix actually
carries windows-latest is the one it is not on. That is defensible as future-proofing --
if the runner image or the matrix ever changes, it is already in place -- but something that
reached the local run (a pytest-level setting, or a line in the contributor docs) is what
would answer the sentence the description makes.

Nit 2 -- the in-file comment describes a matrix that does not exist

ci.yml:91-95 says "Setting this at the job level means pytest, coverage, and any subprocess
all run in UTF-8 mode regardless of the runner OS". That job has one OS. The description says
"On Linux this is a no-op", which is the accurate version -- but the description does not
live in the file, and the comment is what the next person editing this job reads. Same two
readers, two different facts.

Smaller, in the same comment: "the stock shell on a GBK-locale Windows machine reads and
writes files in the locale code page" -- the shell is not what decides this; Python's
locale.getpreferredencoding(False) is, which is why PYTHONUTF8 fixes it and a different
shell would not.

Stated, not filed

Nothing pins the three assignments. Removing all three (and the two env: keys they leave
empty, so the file still parses -- I checked with yaml.safe_load, my first attempt made it
malformed and I threw that result away) leaves every test that reads a workflow green:
395 passed and 14 passed. The control that this is a real measurement and not a suite
that never opens the file: changing the 5.1 step's shell: powershell to pwsh turns
test_the_ci_gate_runs_the_windows_install_under_both_powershells red, 1 failed, 46 passed.
Not filed as its own point because CI env vars are not normally pinned here -- it is only
worth a line because this PR does add a tripwire for its other ci.yml change.
PYTHONUTF8 on the unit job also reaches the coverage data write, as your description
says. Coverage files are the tool's own format and I did not find a path where the code page
could change them, so I am not filing anything there.
I could not build a non-UTF-8 interpreter on this machine to demonstrate the failing decode
end to end: macOS enables UTF-8 mode for the C locale by itself, so my control could not be
moved. The cp1252/GBK results above are bytes.decode(...) on the real files, not a run of
the suite under those locales, and I am saying so rather than implying I ran it.

Not covered

The delta is CI configuration; no runner was exercised. Everything above about what a job
would do is read off the parsed workflow and the repo's bytes, not off a build.

gloryfromca

No blockers; this can merge as far as I am concerned.

Reviewed the delta from 042a7e58819a and the resulting full diff. Moving UTF-8 mode into the Makefile's pytest targets makes the setting reach the supported local and CI test entry points, while the installer-job setting reaches the Windows installer subprocesses. The changes neither weaken coverage nor alter the PowerShell redirect-resolution paths.

I rechecked the repository rules, workflow and Makefile callers/history, backward compatibility, test integrity, and architecture constraints; no domain or layer boundary is affected. Verification: the focused installer/onboarding suite passed with 395 tests under PYTHONUTF8=1; git diff --check, the source-language gate, and the large-file gate passed. The Windows installer and self-upgrade jobs also pass on this head.

0xKT · 2026-10-06T17:14:26Z

Not a blocker. Round three on 00d91408. Both round-two nits are closed and the move is

Severity of this one finding, not a verdict on the pull request. The blocking findings from this pass are review threads on the changed files; GitHub renders those collapsed, as a file name with no text.
the right shape. One nit left, and it is about where the remaining gap sits rather than
about the move.

Both nits closed

The variable is now where the locale can differ. PYTHONUTF8 is off unit and
trajectory -- the two jobs whose matrix is ubuntu-latest only, where it could not change a
decode -- and off the windows-upgrade step; it is now job-level on installer, the one job
whose matrix actually carries windows-latest. Read off the parsed workflow, not the diff.

And it now reaches a local run. make -n expands all three pytest targets with it:
PYTHONUTF8=1 uv run ... pytest -q, TERM=dumb PYTHONUTF8=1 uv run ... pytest -q --cov=...,
and the same for coverage-shard. The unit job still runs make coverage-shard
(ci.yml:151), so CI keeps it through the Makefile rather than through a second declaration
-- one definition, two readers, which is what the previous shape was missing.

Dropping it from windows-upgrade costs nothing measurable. That job runs
pytest --noconftest tests/integration/test_cli_upgrade_real_uv.py on windows-latest, and
parsing that file (not grepping it) shows all six of its text-I/O calls already name
encoding="utf-8" -- lines 27, 52, 105, 106, 148 and 205. Control that the parse can see a
missing one: the same walk over tests/_ppt_engine_skill_claims.py reports 17 encoding-less
calls.

Gates: gates.sh ruff/lint-imports/commit/large/lang rc=0; make check-large-files rc=0;
the five files that read a workflow or the Makefile, 409 passed.

Nit -- the one documented way to run the suite is the path that stays exposed

The Makefile comment says it plainly, and it is right: "a bare uv run pytest does not go
through this file and stays exposed." The thing worth a line is which path that is.
AGENTS.md:393 instructs: "always run tests via uv run pytest ..., never bare pytest"
(the line ends with a section cross-reference I am not reproducing, since this comment is ASCII), and AGENTS.md:257 gives uv run pytest <relevant tests> -x as the rebase step.
So the repo's own documented invocation -- the one an AI-assisted contributor is told to use,
which is the audience your third topic names -- is exactly the one the Makefile cannot reach.

Not blocking, and not a regression: before this commit there was no local protection at all,
so this strictly improves. It is incomplete rather than wrong. Something that travels with the
interpreter rather than with the caller -- a pyproject.toml pytest-env entry, or a
conftest.py that refuses to run a non-UTF-8 interpreter with a sentence saying why -- would
close it for every invocation, including the documented one.

Stated, not filed

Still nothing pins the variable: removing it from both new homes (the Makefile definition,
the three target exports, and the installer job env, with the YAML re-parsed so the mutant
stays valid) leaves 424 passed across the six files that read either file. The control
that those files really are read: renaming one --cov= spec in the Makefile turns
test_coverage_gate.py::test_the_diff_gate_claims_only_trees_that_coverage_measures red,
1 failed, 14 passed. Same observation as last round, repeated because the homes changed;
I am not filing it twice.
I could not demonstrate the mode flipping on this machine -- macOS enables UTF-8 mode for
the C locale on its own, so sys.flags.utf8_mode is 1 either way here. What I did measure is
that the variable reaches the child's environment (PYTHONUTF8=1 present vs absent); the
decode behaviour itself is reasoned from that, and I would rather say so than imply I ran it
under cp936.

gloryfromca

No blockers; this can merge as far as I am concerned.

Reviewed the delta from 00d914083e05 and rechecked it against the resulting full diff. The conftest guard closes the documented bare uv run pytest gap, fails before locale-dependent test I/O can produce misleading results, and leaves the --noconftest Windows-upgrade path unchanged. It does not affect application runtime or cross an architecture boundary.

I rechecked repository rules, callers and history, backward compatibility, and test integrity. Verification: the focused installer/onboarding suite passed with 395 tests under UTF-8 mode; a forced non-UTF-8 interpreter failed with the intended actionable RuntimeError; the documented python -X utf8 -m pytest remedy restored successful collection of all 47 installer tests. git diff --check, source-language, and large-file gates passed, as did the Windows installer and self-upgrade CI jobs.

LivXue

Code review -- read as request-changes

Ten independent finder angles, two verifier agents, and a gap sweep ran against this PR. Several claims were reproduced empirically on the PR tree. The core fix (-ErrorAction Ignore in Resolve-RavenLatestVersion for PowerShell 5.1) is correct: both old and new code surface identical failure UX on network errors, the 5.1/pwsh shell split is the right mechanism with no simpler unified alternative, and the new live CI step empirically validates the 5.1 behavior on every run.

The substantive findings are in the UTF-8 test-infrastructure half of the PR.

Must fix before merge

Makefile:20 -- check-core-wheel (the fourth pytest recipe, part of make ci) invokes pytest without the PYTHONUTF8 prefix the other three targets got. On a GBK-locale Windows machine make ci passes lint/test/build, then dies at conftest import in check-core-wheel. The comment at line 18 also mis-states the situation ("The three pytest targets export it below" -- four pytest recipes exist). A single export PYTHONUTF8 := 1 covers every recipe line including this one.
tests/conftest.py:19 -- The UTF-8 gate only accepts 'utf-8'/'utf8' and rejects 'cp65001'. A Windows machine with the system-level "Beta: Use Unicode UTF-8 for worldwide language support" option (ACP=65001) is genuinely UTF-8 for all default text I/O (codecs.lookup('cp65001').name == 'utf-8'), but locale.getpreferredencoding(False) returns 'cp65001' and the gate hard-fails every uv run pytest.
tests/conftest.py:19 -- Hard-failing the entire suite at conftest import for every non-UTF-8 run path is disproportionate to the one observed failure (a workflow-text scan that already got its root-cause encoding="utf-8" fix). The ~790 remaining bare read_text() calls behave identically under cp936 and UTF-8 on ASCII content. This converts "developer on GBK Windows runs the suite" from working-minus-one-test into blocked-at-collection for every entry point that misses PYTHONUTF8 (IDE runners, bare uv run pytest, tox). A pre-commit/ruff gate over tests/ flagging read_text()/write_text()/open( without encoding= catches new offenders deterministically without refusing working environments.

Should fix

tests/test_install_script.py:479 -- Bare re.search(...).group(1) and str.index slicing: a plausible ci.yml refactor (job-level defaults: run: shell:, re-indenting, renaming windows-upgrade) raises AttributeError/ValueError instead of a readable assertion naming which invariant drifted.
.github/workflows/ci.yml:292 -- Job-level PYTHONUTF8: "1" on the installer job means the freshly installed raven --version smoke now runs in UTF-8 mode, whereas every real user on a legacy-code-page Windows machine runs it under cp1252/cp936. If raven's --version path ever does locale-default text I/O, the gate green-lights a build that fails for exactly the users the PR's docs section addresses.
tests/test_install_script.py:476 -- The new gate test byte-duplicates the ci.yml read + installer-job slice that the adjacent test already has at lines 460-462 (same read_text, same " installer:" anchor, same "\n windows-upgrade:" end anchor). Extract a module-level helper.

Nonblocking observations

Makefile:20 -- PYTHONUTF8 = 1 as a make variable plus per-line PYTHONUTF8=$(PYTHONUTF8) prefixes is indirection for a constant that is never overridden. A single export PYTHONUTF8 := 1 covers every recipe line (including check-core-wheel, which the per-line prefixing missed).
.github/workflows/ci.yml:354 -- The new shell: powershell step runs a second full install.ps1 sequentially after the identical pwsh step, serially doubling the Windows leg's install wall time. A matrix axis would run both shells in parallel; shared UV_CACHE_DIR amortizes downloads but not the install/link/verify pass.
tests/test_install_script.py:480 -- The two-shell gate test pins UV_TOOL_DIR distinctness but never pins UV_TOOL_BIN_DIR (or RAVEN_HOME), although the version probe that makes the gate meaningful is $env:UV_TOOL_BIN_DIR\raven.exe --version. A future edit sharing one UV_TOOL_BIN_DIR between steps makes the ps51 probe answer for the pwsh install while CI stays green.
.github/workflows/ci.yml:261 -- The trajectory job runs uv run pytest directly with no PYTHONUTF8 anywhere -- the exact bare-invocation exposure the PR's own Makefile comment calls out ("a bare uv run pytest does not go through this file and stays exposed"). Benign today (ubuntu-latest, C.UTF-8); one job-level env: line closes the asymmetry.
Commit 00d9140 body -- References internal commit hash d57ad789 and "this branch", which AGENTS.md section 3.3's don't-add list forbids in commit messages ("internal commit-hash references / temporary branch names -- docs/PRs describe the present state only"). Describe the change by its subject instead.

Verified and excluded

pwsh-7 -ErrorAction Ignore swallowing network errors: new and old code produce identical failure UX; no functional regression.
Shared UV_CACHE_DIR between sequential steps: uv cache is content-addressed; no lock contention in serial execution.
The conftest gate killing non-pytest tooling entry points: repo-wide grep confirms all from tests.conftest import hits are pytest-collected test modules, so the guard's blast radius is confined to the suite as intended.

install.ps1 finds the latest release by requesting the release page without following its redirect and reading the Location header off the exception that raises. Only PowerShell 7 raises there. Windows PowerShell 5.1, the shell every Windows ships with, returns the unfollowed redirect as the response and reports the exceeded redirect count as an error with no response attached, so the lookup always came back empty and every one-line install from the stock shell stopped at "Could not resolve the latest Raven release wheel from GitHub". The request now ignores that error and reads the Location header from the response; PowerShell 7 still raises and still goes through the catch. The installer CI job ran install.ps1 under pwsh alone, which is how this went unnoticed. It now also runs the piped install under Windows PowerShell 5.1, into its own tool, bin and home directories. The short URL raven.evermind.ai/install.ps1 answers with a 308, which Windows PowerShell 5.1 does not follow either. The README, the docs site quick start and the release notes template now name the error it prints, (308) Permanent Redirect, and send readers to the direct URL when they see it. Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>

Three tests failed when the suite ran on a Chinese-locale Windows checkout. CI runs the unit tests on Linux only, so it never saw them. The workflow scan read each file in the locale encoding, GBK there, and stopped on the UTF-8 in release.yml. It now reads UTF-8, as the rest of the file already does. The font and LibreOffice harnesses run install.sh code under sh. On Windows that sh is Git Bash, whose curl sits outside /usr/bin and whose paths mix separators, so two harness tests failed and others passed for the wrong reason: the digest-mismatch test passed because curl was missing, not because of the digest. They now carry the POSIX-only skip the other sh-driven tests already had, shared as one marker. Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>

Python's locale-dependent text I/O uses the system code page when no explicit encoding= is passed. On a GBK-locale Windows checkout, any test that reads or writes a file containing non-ASCII bytes will fail or silently corrupt data. The class of bug was found in this PR when test_no_workflow_step_enters_the_removed_page_directory stopped on the UTF-8 in release.yml; there are ~790 read_text() call sites without encoding in tests/, and no PYTHONUTF8 anywhere in the test infrastructure, so the next GBK-console developer would hit the next one. Set PYTHONUTF8=1 on every CI job that runs pytest: the unit shard matrix (which also runs the coverage steps), the trajectory regression replay, and the Windows self-upgrade job. On Linux this is a no-op. On Windows it forces every open()/read_text()/write_text() to use UTF-8, making the behaviour identical to what Linux CI already sees. tests/conftest.py is not touched: local developers can set the same variable, or add the flag to their pyproject.toml if needed, but that is a follow-up. Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>

The variable set by the previous commit sat on jobs whose matrix never carries a Windows runner, so it could not change anything, and the job actually running pytest on Windows -- the installer job -- had none of it. Move it there, with an honest comment. The bot also pointed out that a local `uv run pytest` on a GBK checkout never passes through CI, so a developer on the shell this is all about is exactly as exposed before the commit as after. Export PYTHONUTF8 from the Makefile's three pytest targets instead; the bare `uv run pytest` stays exposed, which is a separate follow-up. The test_cli_onboard_commands.py fix for the encoding-less read_text() in the workflow scan is part of test(*): skip the sh harnesses on windows, which is already in this branch (d57ad78). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>

AGENTS.md tells contributors to run 'uv run pytest', and that caller does not read the Makefile, so the Makefile exports shipped in the previous commit left the documented local entry point exactly as exposed as before. tests/conftest.py is the one piece of code every conftest-loaded run executes; make it stop a non-UTF-8 interpreter at import with a message naming 'PYTHONUTF8=1' and '-X utf8' as the fix. The gates still pass on the stock Windows shell: on a GBK-locale checkout a bare 'uv run pytest' raises RuntimeError ('cp936'), while any invocation with 'PYTHONUTF8=1' runs the same 45 passed / 13 skipped installer and onboarding tests as before. --noconftest runs (the windows-upgrade CI job) do not load conftest at all, and that job's tests name encoding explicitly, so they are unaffected. Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>

check-core-wheel runs pytest under tests/conftest.py too, and it was the one pytest recipe without the PYTHONUTF8 prefix, so on a GBK checkout `make ci` passed test-python and then stopped at the conftest guard. One exported assignment reaches every recipe, targets added later included, and the comment no longer counts them. Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>

With the system-wide UTF-8 option on, Windows reports its code page as cp65001, which is the UTF-8 codec under another name, and the guard compared names, so it refused every run on a machine that already decodes UTF-8. It now compares codecs. The new tests pin the predicate on the names an interpreter reports, and drive the refusal end to end in an ASCII-locale child so that deleting the guard fails a test. Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>

The installer job runs no tests. After checkout its steps install raven and run it: install.sh or install.ps1 installing the latest release, and the installed raven answering --version. A job-level PYTHONUTF8 changed nothing but how that raven runs, and users on a legacy code page run it without UTF-8 mode, which is the way this gate exists to run it. Both PowerShell installs passed on windows-latest without the variable before it was added here. Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>

Both CI gate tests read ci.yml and cut the installer job out of it with the same two anchors, so renaming either job meant editing both in step. They now share _installer_job(). Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>

The version check runs the raven.exe in UV_TOOL_BIN_DIR, yet the gate test pinned only UV_TOOL_DIR as distinct, so two steps sharing a bin or home directory passed it while one install could answer for the other. It now pins all three, as the ci.yml comment above the step promises, and a step missing a setting fails naming the pattern and the step rather than with an AttributeError. Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>

LivXue · 2026-10-08T08:26:06Z

Round summary for the review above, by its item numbers.

Fixed, each reproduced before the change, one commit per finding:

1 and 7, make check-core-wheel without PYTHONUTF8 and the per-recipe prefixes: e0244e7
2, cp65001 refused by the guard: 8091596
5, PYTHONUTF8 on the installer job: 5d6b430
6, the duplicated job slice: 4b7418a
4 and 9, opaque lookups and the unpinned bin and home directories: 1b23b6a

Declined, with the measurement in each thread:

3: the guard stays. On main under a GBK locale, 87 tests fail that pass under UTF-8 and a 418-test module does not collect, so the run it refuses is not one test short.
8: the second install costs 14 s of a 73 s job.
10: the trajectory job is ubuntu-only, and the guard turns a move off it into a loud failure naming the fix.

11: left as is. The squash takes this PR's description as the commit body, so per-commit bodies do not reach main, and that one is not being rewritten.

The branch is also rebased onto main now that #861 has merged. The one conflict was in tests/test_install_script.py; the sh-driven tests #861 added (its two Node harness tests and test_a_download_that_cannot_be_hashed_is_not_installed) now carry @POSIX_SH_ONLY like their neighbours. The description is updated to match, including the Linux verification for this head.

gloryfromca

No blockers; this can merge as far as I am concerned.

Reviewed the commits since 5743cdab84de and the full rebased github/main...HEAD diff. The rebase cleanly incorporates #861 without carrying its installer-integrity changes in this PR's diff. The new fixes cover every Make recipe through one exported PYTHONUTF8, accept Windows cp65001 by codec identity, keep the installer smoke representative of real user environments, centralize the workflow slice, improve failed-pattern diagnostics, and pin independent tool, bin, and home directories for both PowerShell installs. The declined items do not leave a concrete failure in the current scope.

I rechecked repository rules, callers and history, PowerShell compatibility, test strength, and architecture constraints; no domain or layer boundary is affected. Verification: the conftest/installer/onboarding suite passed with 410 tests and 4 PowerShell-dependent skips; the core-wheel smoke passed; git diff --check, source-language, and large-file gates passed. The Windows installer, Windows self-upgrade, lint, repository, and other completed CI checks are green on this head.

LivXue

Self-check sweep after the round-1 fixes. No Must fix items; the suite reads clean end to end. Five items remain, ordered by weight. This is a COMMENT review; read the first two as request-changes.

Should fix

tests/test_install_script.py:747 -- the both-powershells mirror test pins shell name and three directory names, but not the step's if: guard, RAVEN_MINIMAL / RAVEN_NO_LAUNCH, or the run-body version probe. A dead or half-configured ps51 step keeps the test green while the gate stops exercising 5.1.
.github/workflows/ci.yml:381 -- the windows-upgrade job runs pytest with --noconftest on windows-latest (cp1252), and pyproject.toml norecursedirs = ['tests/integration', ...] confirms tests/integration is excluded by default. The Makefile comment at lines 16-20 claims tests/conftest.py refuses that run instead -- that claim is false for this lane. Today the file is encoding-insensitive so nothing breaks, but the invariant is silently unenforced on exactly the platform the guard was built for.

Nonblocking

tests/test_install_script.py:729 -- the same Get-Content install.ps1 -Raw | Invoke-Expression string is pinned twice in one file. The both-powershells comprehension plus sorted(...) == ['powershell', 'pwsh'] already fails the moment the string disappears from either step.
.github/workflows/ci.yml:342 -- the 4-line comment duplicates the test docstring at tests/test_install_script.py:735-741 nearly verbatim.
tests/conftest.py:29,35 -- getpreferredencoding(False) is called twice in the guard; naming it once above the if removes the chance a future edit changes only one call site.

Verified clean, not findings

Makefile line-21 export PYTHONUTF8 := 1 is inherited by check-core-wheel via uv run; the conftest guard still fires there.
cp65001 ("UTF-8 system code page") passes the guard's codecs.lookup(name).name == 'utf-8' check correctly; that branch is already handled by commit 8091596 in this PR.
With loCPATH=zh_CN.GBK + PYTHONUTF8=1 the guard's getpreferredencoding(False) reports 'utf-8' (Python >= 3.15 deprecates this API and reports UTF-8 mode), so a Windows user who follows the message's own fix is not re-blocked. Verified with an isolated conftest run under GBK: 1 passed.
The pwsh 7 catch path around Get-RavenLatestRelease is live: a local pwsh 7.6.5 against a 302-stub server throws into the catch on every ErrorAction; the catch is the only place that reads the Location header. The stale "would regress pwsh 7" scenario from the earlier 07:24 note was wrong.

Happy to push a follow-up commit collapsing the three nits into one commit if that lands easier than three threads.

ZuyiZhou

Approved at 1b23b6a. Read the diff. The install.ps1 change is -ErrorAction Ignore on the release-page request: Windows PowerShell 5.1 then returns the unfollowed redirect and Location is read off the response, while PowerShell 7 still raises into the existing catch, and the Location is matched against the same stable-tag pattern. The Windows installer job at this head ran the new 5.1 step (shell: powershell, own tool, bin and home directories) and it passed, so the fix is exercised under the stock shell on every run. The conftest guard compares codecs, so cp65001 passes, it names PYTHONUTF8=1 in the error, and the Makefile export covers every recipe including check-core-wheel. The POSIX_SH_ONLY marker only skips harness tests on Windows. Not rerun here: the test suites, and install.ps1 on a Windows host. Non-blocking: the open thread on tests/test_install_script.py:747 (the mirror-steps test could also pin the if: guard, RAVEN_MINIMAL / RAVEN_NO_LAUNCH and the version probe); it is still unresolved, so the every-thread-resolved rule keeps the merge button off until it is answered.

gloryfromca reviewed Oct 5, 2026

View reviewed changes

gloryfromca reviewed Oct 6, 2026

View reviewed changes

gloryfromca reviewed Oct 7, 2026

View reviewed changes

LivXue commented Oct 8, 2026

View reviewed changes

LivXue and others added 10 commits October 8, 2026 07:50

LivXue force-pushed the fix/ps51_release_lookup branch from 5743cda to 1b23b6a Compare October 8, 2026 08:25

gloryfromca reviewed Oct 8, 2026

View reviewed changes

LivXue commented Oct 8, 2026

View reviewed changes

Comment thread tests/test_install_script.py

Comment thread .github/workflows/ci.yml

Comment thread tests/test_install_script.py

Comment thread .github/workflows/ci.yml

Comment thread tests/conftest.py

ZuyiZhou approved these changes Oct 8, 2026

View reviewed changes

LivXue merged commit 3c88206 into main Oct 8, 2026
49 checks passed

LivXue deleted the fix/ps51_release_lookup branch October 8, 2026 09:31

0xKT mentioned this pull request Oct 9, 2026

fix: Model request failed on Windows #119

Closed

Conversation

LivXue commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type

Verification

Risk

Related Issues

Uh oh!

gloryfromca left a comment

Choose a reason for hiding this comment

Uh oh!

0xKT commented Oct 5, 2026

Uh oh!

gloryfromca left a comment

Choose a reason for hiding this comment

Uh oh!

0xKT commented Oct 6, 2026

What I checked and found right

Nit 1 -- the measure is in the one place the developer it names never reads

Nit 2 -- the in-file comment describes a matrix that does not exist

Stated, not filed

Not covered

Uh oh!

gloryfromca left a comment

Choose a reason for hiding this comment

Uh oh!

0xKT commented Oct 6, 2026

Both nits closed

Nit -- the one documented way to run the suite is the path that stays exposed

Stated, not filed

Uh oh!

gloryfromca left a comment

Choose a reason for hiding this comment

Uh oh!

LivXue left a comment

Choose a reason for hiding this comment

Code review -- read as request-changes

Must fix before merge

Should fix

Nonblocking observations

Verified and excluded

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LivXue commented Oct 8, 2026

Uh oh!

gloryfromca left a comment

Choose a reason for hiding this comment

Uh oh!

LivXue left a comment

Choose a reason for hiding this comment

Should fix

Nonblocking

Verified clean, not findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ZuyiZhou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

LivXue commented Oct 5, 2026 •

edited

Loading